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Abstract. In the field of computer vision, a crucial task is the detec- 
tion of motion (also called optical flow extraction). This operation allows 
analysis such as 3D reconstruction, feature tracking, time-to-collision 
and novelty detection among others. Most of the optical flow extraction 
techniques work within a finite range of speeds. Usually, the range of 
detection is extended towards higher speeds by combining some multi- 
scale information in a serial architecture. This serial multi-scale approach 
suffers from the problem of error propagation related to the number of 
scales used in the algorithm. On the other hand, biological experiments 
show that human motion perception seems to follow a parallel multi- 
scale scheme. In this work we present a bio-inspired parallel architecture 
to perform detection of motion, providing a wide range of operation and 
avoiding error propagation associated with the serial architecture. To 
test our algorithm, we perform relative error comparisons between both 
classical and proposed techniques, showing that the parallel architecture 
is able to achieve motion detection with results similar to the serial ap- 
proach. 
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1 Introduction 

The visual capabilities in humans have motivated a large number of scientific 
studies. The performance to perceive and interpret visual stimuli in different 
species including humans is outstanding: the wide range of tolerance to different 
illumination and noise levels are just a few characteristics that we are aware of, 
but that are still barely understood. 

An important aspect in visual processing is the perception of motion. Mo- 
tion is a key step in several computer vision tasks such as 3D reconstruction, 
feature tracking, time-to-collision estimation, novelty detection, among others 
PP. Motion is also one of the features that many species can perceive from the 
flow of visual information, and its detection has been observed in a large num- 
ber of animals 0, from invertebrates to highly evolved mammals. From optical 
engineering and experimental psychology we already know the main features of 
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human motion discrimination [3J. In this work, we are particularly interested 
in taking inspiration from biology in order to design a parallel algorithm with 
similar discrimination capabilities as obtained by classical serial architectures. 

Our work begins by presenting an overview of techniques to detect motion 
in machine vision to continue with the available experimental results and their 
procedures in human psychophysics. Section 3, presents our algorithm for speed 
detection with which we perform our simulations, and compare with experimen- 
tal data. The results are analyzed in section 4, and in the last two sections, we 
present the discussion and conclusions about our work. 



2 Overview 

In this work we are interested in the detection of motion, specifically in the 
coding and retrieval of speed (v), and in the link between the idea of selecting 
a range of speed to work with, and providing wider ranges of discrimination 
as observed in human psychophysics experiments @]. We focus on two features: 
the multi-scale architecture of the speed detection, and the relation between the 
number of multi-scale levels and the range of speeds the system is sensitive to. 

2.1 Motion detection in computer vision 

The detection of motion is a widely used operation in computer vision. Com- 
monly called "optical flow extraction" , the main objective is to assign a vector 
v = (u, v) to each frame pixel from a given sequence of frames (at least two). In 
this section, we explain the basic technique to increase the motion range of an 
optical flow extraction which the method is sensitive to. We ground our expla- 
nation on the well-known Lucas & Kanade's method 0[l] (the basic multi-scale 
technique similarly applies to other methods for optical flow extraction). 

Optical flow Many optical flow extraction methods are based on the initial 
assumption of brightness conservation, that is, 

dl(x,y,t) 81 81 81 „ 

~dt = d^ U + !h, V + ~di = (1) 

where v = (u, v) is the velocity vector. A well known technique following this 
approach is the Lucas & Kanade algorithm [5] , that minimizes the following cost 
function in a small fixed region f2, i.e. 



arg min < W 2 (x) 
Lett 



dl 

V/(x,t)-v + — (x,t) 



(2) 



where W is a two-dimensional Gaussian function used to give more importance 
to the central points and fl is a square region of a few pixels. This minimization 
estimates v with sub- pixel precision after a few iterations. This method achieves 
good optical flow extraction in regions where |W(x,i)| > 0, such as corners [5J. 
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Serial multi-scale optical flow The Lucas & Kanade method for optical flow 
extraction considers a small region Q. The use of this region Q is not particular to 
this method: it is used in most algorithms pQ. As the computation is performed 
in small windows, the detection of motion is constrained to detect speeds up 
to uj pixels per frame, where u> stand for the diameter of fl. To overcome this 
limitation, a multi-scale representation of the images can be performed, usually 
by considering Gaussian pyramids [7]. A Gaussian pyramid representation of 
an image is computed by recursively smoothing (using a Gaussian kernel) and 
sub-sampling the original image. In this way, the original image is represented 
by a set of smaller images. The representation at scale level Z = is the original 
image itself. The image at level I is obtained by sub-sampling a filtered version 
of the image at level I — 1 with a downsampling factor equal to 2. Thus, the size 
of the image at each level I is TV; = A^_i/2 with I = 1, 2, . . . , (L — 1), where L is 
the number of levels of the representation. 

In the serial multi-scale optical flow estimation, speed is computed by sequen- 
tially projecting the estimation obtained at level I to level I — 1, until level / = 0. 
There are complex strategies for computing the optical flow with a multi-scale 
approach [6]. A simple solution for optical flow computation is implemented in 
the widely used computer vision library OpenCV [3 [7] . In this case, the multi- 
scale estimation starts from the highest level (I = L — 1) and it propagates to 
the next one: 

V;_i = 2 * V; + di-i(vi) (3) 

where di-\ is the estimation of velocity at level I — 1 after projecting the estima- 
tion vi by warping the image by —vi at level I — 1. Computing the optical flow 
from the highest level and then projecting the solution to the lower level [5j [7] 
increases the range of detectable speeds. This range is wider when more scales 
are used. On the other hand, the sequential projection between levels also prop- 
agates the error introduced at each level. Thus, in terms of precision, increasing 
the number of scales in the representation increases the error introduced in the 
estimation. 

2.2 Biological elements 

This section sketches out the current experimental knowledge in biology, focusing 
on studies of speed coding in the human brain [9] and on higher level descriptions 
of speed discrimination from experimental psychophysics [3J I10[ lllj . 

Parallel architecture In the human brain the main area that is responsible 
for coding different speeds is area MT [2]. It is located in the occipital region 
(back of the head). Neurons in this area are selective to stimuli moving at a given 
speed |12) . Their spatial organization is retinotopical .13] : each neuron has a re- 
duced visual field, and neurons who share the same local visual field are grouped 
together in some macro-column that contains cortical columns that are selective 
for different orientations. This configuration allows a complete mapping of the 
visual field with a group of cortical columns that codes for all possible directions 
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of local motions. The spatial organization is less known with respect to the speed 
selectivity. Nevertheless, it has been found that (1) the average detected speed in- 
creases with eccentricity (with respect to the retinotopical organization of MT), 
(2) similar speeds are detected by neurons closer than for distinct speeds, and (3) 
for each eccentricity, there are neurons for different speeds [13] . The interactions 
between different units is not completely understood, but there is evidence that 
units sensitive to different speeds could be coding a range of speeds in parallel 
[9]. It has been observed that the range of detectable speeds is not uniformly 
covered |12) , but in this work we are interested in the simultaneous existence of 
speed selective units in MT that could be accountable for a parallel architecture 
dealing with different speeds. 

Speed discrimination In the work of McKee et al. [3J, two subjects were 
exposed to several stimuli, one of those being a horizontal scaled single bar 
vertically moving at different eccentricities. The goal of this experiment was to 
determine the minimal relative detectable variation in speed for every subject 
with the sight fixed at a certain location and for each stimuli eccentricity. 

It is important to mention how this was actually measured, because the sub- 
ject cannot assign a precise velocity at each location. Instead, given a reference 
velocity, the subject was asked to indicate whether the next presented stimuli 
moves faster or slower. The minimal detectable variation was then statistically 
inferred. Related experiments were performed by others [Il[l0j[TT], showing that 
the measurements are not affected by different contrast conditions, and that they 
do not depend on binocular or monocular sight. 

The described experiments study the speed discrimination at several eccen- 
tricitie^l, see Fig. [TJ In this work we are interested in each one of these eccentrici- 
ties and their related discrimination properties, and not in the relations between 
different eccentricities. In order to model these discrimination functions, we need 
to generate a given discrimination percentage in a range of speed [i>i,i>2]- We 
also point out that the left side of the experimental curves, see Fig. [1] is related 
to the eccentricity but the same idea holds: for each eccentricity there is a wide 
range of speed discrimination, where the relative error (rather than the absolute 
error) remains stable (5%-15%). 

3 Proposed parallel multi-scale speed detection 

Multi-scale speed detection is based on the fact that a particular speed detection 
algorithm can be used to estimate slower speeds at lower levels and to estimate 
faster speeds at higher levels. This information is used in the above described 
serial multi-scale optical flow algorithm to detect speeds in a wide range of 
velocities by projecting the information at level I + 1 to estimate speed at level I, 
i.e. in a serial manner. As it is described in [121 [9]. it seems that human motion 
perception is based on a parallel multi-scale scheme. Based on this idea, the speed 
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Fig. 1. Weber fraction (minimum threshold of perceived change in speed or Av/v) as 
function of velocity, for different eccentricities in the human subject B.D.B. (condition 
2), in the differential motion experiment by [4] (reproduced from their work), each 
curve is at eccentricities 0,10,20,30 and 50, respectively (left to right). The speed axe 
is in logarithmic scale. 



detection algorithm proposed in this paper estimates the speed by combining 
the information computed at each level independently, i. e. using the multi-scale 
information in a parallel manner. In this case, there is no error propagation 
on the computation of speeds at each level because it does not depend on the 
estimation performed for other levels. At each level I, we compute speeds using 
the optical flow estimation algorithm described above, see subsection 12.11 As 
explained before, this choice does not bias our results, since our work is to 
provide a bio-inspired parallel speed detection instead of the standard serial 
approach, for any optical flow extraction method. 

As expected, the speed detection algorithm estimates speed with a certain 
error at each multi-scale level I. The confidence in the estimation of speed v r at 
level I, denoted as ki(v r ), can be defined as 



ki(v r ) = 1 



(4) 



where v r is the magnitude of the object's real speed (v r — ||vy||) and v e is 
the magnitude of the average estimated speed on the object pixels location. It 
can be noted, that this computation only takes into account the magnitude of 



the speed, ignoring its direction. Figure 2(a) shows the confidence ki(v r ) for 



three different multi-scale levels. These distributions were computed using an 
input image sequence containing an object moving at different speeds in a range 
from 0.5 pixels per frame to 20 pixels per frame. To statistically determine the 
confidence at each level I and speed v r , the experiments were carried out using the 
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input image sequence with several realizations of Gaussian white noise, then the 
resulting confidence ki(v r ) is computed as the mean value of the ones obtained 
in the experiments. Figure 3(a) shows two frames of an input image sequence 
used in the experiments. In this sequence the object is moving at 10 frames per 
pixel in the bottom-right direction. 




Fig. 2. Confidence distribution ki for different levels I. (a) Experimental distribu- 
tions ki. (b) Approximated distributions k\. 
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(a) (b) 

Fig. 3. In (a) one frame of an input image sequence used in the experiments is depicted 
(v = 10 bottom-right direction), (b) is the obtained optical flow using the proposed 
parallel multi-scale algorithm. Note that optical flow image was zoomed around the 
object position. 



As it may be seen in Figure 2(a) a particular speed v r can be detected at 
several multi-scale levels but with different confidence values. Thus, the current 
speed could be estimated by taking into account the speeds computed at each 
level I and their associated confidence values k\. For that reason, the experimen- 
tal distributions depicted in Fig. 2(a) have to be approximated by a closed-form 
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equation. In this work, these distributions are approximated (modeled) as Gaus- 
sian distributions in a semi-log space defined by the following equation 



h(v r ) = exp 



log(iv) - m 



(5) 



Mz = Mo + log(c ) (6) 
m = er (7) 

where fiQ and <7o are the mean and variance of the distribution at level I — and 
c is the scaling factor used in the sub-sampling of the images. The approximated 
distributions ki for I = 0, / = 1 and I = 2 are depicted in Fig. |2(b)| It may be 
seen that a better approximation of the distributions could be obtained using 
a particular set of variables for each level but this would increase the model 
complexity. The approximation of the distributions k\ for each level in Eq. ([5]) 
only depends on [i , &o and c. It may be noted that this approximation allows 
to perform the estimation of speeds using different values of the scaling factor 
c, which is usually set to c = 2, i.e., the case of using Gaussian pyramids for the 
sub-sampling. 

Finally, denoting the detected speed at each level I by v l e , the proposed 
algorithm computes the current speed, using the speed detected at each multi- 
scale level with its associated confidence value fci(||v' II), as 



Ef^CIKIi; 



(8) 



where L is the number of levels used to compute the estimated speed v / . Fig- 



ure 



3(b) shows the obtained optical flow using the proposed parallel multi-scale 
algorithm. The comparison between the experimental confidence distribution of 
the proposed algorithm and confidence distributions for three levels is shown in 
Fig. 0] As expected, the confidence distribution of the parallel multi-scale algo- 
rithm with L = 3 is approximately the envelope of the confidence distributions 
of levels 1 = 0,1 — 1 and I = 2. 



4 Results 



As it was described in subsection 12. 2\ speed discrimination is computed as the 
minimal detectable variation in speed of a particular visual stimuli. In this work, 
a variation in speed, from a given reference speed v bj, of the moving object is 
considered to be noticeable if the following inequality holds 



i±Av 



Vobj 



> a 



(9) 



where v Vobj and v Vobj ±Av obj are the speeds estimated by the algorithm when the 
object is moving at velocities and ± Av bj, respectively, and a is the 
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Fig. 4. Comparison between confidence distribution of the proposed algorithm (par- 
allel) with L — 3 and c = 2, and confidence distributions k t for levels I = 0, 1 and 2 
without projection between them. 




Fig. 5. (a) Discrimination of the proposed algorithm for L = 1, L = 2 and i = 3 
(c = 2). In (b), (c) and (d), the comparisons between the discrimination of the proposed 
parallel (c = 2) and the serial algorithms for L — 2, L — 3 and L = 4, respectively, are 
shown. 
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percentage of variation from the object speed u D y required to consider Av bj as 
detectable. It may be noted in Eq. ^ that a variation in speed is considered 
noticeable if it is detectable when v a bj is both increased and decreased in Av a bj . 
To statistically determine the minimum value of Av bj several experiments were 
carried out using the input image sequence with several realizations of Gaussian 
white noise. Then, the minimal detectable variation in speed, from a given refer- 
ence speed Vobj , is computed as the minimum detectable Av bj obtained in 90% 
of the experiments. 

We summarize our results in Fig. Figure 5(a) shows the discrimination of 
the proposed parallel multi-scale algorithm for different values of L. The range 
of discriminated speeds is enlarged when the number of levels used in the multi- 
scale representation increases. In comparison with the serial multi-scale, our 
method has a similar range of speed discrimination when the same number of 
levels are used, see Fig. 5(b) 5(c) and |5(d)| Considering both mean and variance 
of the discrimination in the range of speeds from 1 to 15 pixels per frame, the 
parallel multi-scale method shows lower values. For the case of L = 3, parallel 
discrimination has mean= 14.1 and variance^ 2.2, while serial discrimination 
has mean= 15.5 and variance= 4.2. This indicates that the proposed parallel 
algorithm presents a better discrimination in this range. 



5 Discussion 

Recent works [SJ [T3] have developed the idea of multi-scale estimation of speed. 
First, Simoncelli [6J proposes a bayesian scheme to compute the error distribu- 
tions and then to estimate the velocity using a Kalman filter through the space of 
scales (not time). This approach builds a far more sophisticated error function, 
but it is still serial. Our work assumes that the error functions are fixed, while 
[6jassumes the error changes with respect to V/(x,i), and this might be impor- 
tant in real-world scenarios. On the other hand, Chey et al. [14] propose that con- 
sidering higher threshold levels for higher scales (scale-proportional thresholds) 
and inter-scale competition could explain human speed discrimination curves. 
We have presented a scheme where the response of each scale regulates the rele- 
vance of the responses of that scale. Since we handle all scales at the same time, 
it corresponds to a notion of threshold and competition. To our knowledge, no 
other work models error functions as Gaussians in the log space (this strengthen 
the idea that detection is not symmetrical), which seems to fit recent recordings 
of motion sensitivity of neurons [9] . 

Finally, about the time complexity of our algorithm. Lets consider the size of 
the image as N, the order of the optical flow algorithm as K (clearly K(N)) and 
the number of scales I. The complexity order of the serial multi-scale algorithm 
is IN + IK, and N + IK for the parallel algorithm. The only difference is in 
the operations involved in the merge of scales. Considering the possible speed- 
up using p processor for the case p = I, then S p = j^m^, what can be also 
written as S p = p. This last equation show us that the degree of parallelism 
(taking one level by processor) achieved by our proposed algorithm is linear. 
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6 Conclusions 

In this work we have presented a parallel multi-scale algorithm to perform the 
estimation of motion using two consecutive images. This method takes bio- 
inspiration from human physiology and psychophysics knowledge in the sense 
that it achieves wide uniform relative discrimination properties by using evenly 
spaced logarithmic scales, and it gives results in constant time as a function of 
scales. With respect to the classical serial multi-scale optical flow algorithm, the 
error propagation among scales appears less important for our proposed algo- 
rithm in terms of relative discrimination. We now explore the idea of using more 
biologically plausible methods of optical flow extraction and the integration with 
a foveated topology. 
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